System for creating microscopic digital montage images

ABSTRACT

An imaging apparatus. The imaging apparatus may find an area in which a specimen is present, then focus on the specimen and capture images of the specimen during continuous stage motion.

RELATED APPLICATIONS

The present application is a continuation of U.S. patent applicationSer. No. 11/430,721, filed May 9, 2006, which in turn is a continuationof U.S. patent application Ser. No. 09/919,452, filed Jul. 31, 2001,which in turn is a continuation-in-part of U.S. patent application Ser.No. 09/757,703, filed Jan. 11, 2001, U.S. patent application Ser. No.09/758,037, filed Jan. 11, 2001, and U.S. patent application Ser. No.09/788,666, filed Feb. 21, 2001, all of which are currently pending andassigned to the assignee of the present invention.

BACKGROUND OF THE INVENTION

Laboratories in many biomedical specialties, such as anatomic pathology,hematology, and microbiology, examine tissue under a microscope for thepresence and the nature of disease. In recent years, these laboratorieshave shown a growing interest in microscopic digital imaging as anadjunct to direct visual examination. Digital imaging has a number ofadvantages including the ability to document disease, share findings,collaborate (as in telemedicine), and analyze morphologic findings bycomputer. Though numerous studies have shown that digital image qualityis acceptable for most clinical and research use, some aspects ofmicroscopic digital imaging are limited in application.

Perhaps the most important limitation to microscopic digital imaging isa “subsampling” problem encountered in all single frame images. Thesub-sampling problem has two components: a field of view problem and aresolution-based problem. The field of view problem limits aninvestigator looking at a single frame because what lies outside theview of an image on a slide cannot be determined. The resolution-basedproblem occurs when the investigator looking at an image is limited toviewing a single resolution of the image. The investigator cannot “zoomin” for a closer examination or “zoom out” for a bird's eye view whenonly a single resolution is available. Significantly, the field of viewand resolution-based problems are inversely related. Thus, as oneincreases magnification to improve resolution, one decreases the fieldof view. For example, as a general rule, increasing magnification by afactor of two decreases the field of view by a factor of four.

To get around the limitations of single frame imaging, developers havelooked at two general options. The first option takes the form of“dynamic-robotic” imaging, in which a video camera on the microscopetransmits close to real time images to the investigator looking at amonitor, while the investigator operates the microscope by remotecontrol. These systems have been used successfully in initialtelepathology collaborations by allowing a distant consultant to viewthe specimen without the delays and losses associated with sending thephysical slide to the consultant for review, and by allowing theconsultant to view the entire slide, not just a few static imagescaptured by the initial user.

However, these systems may not lend themselves to significantcollaborations, documentation or computer based analysis. To besuccessful, remote transmission requires lossy video compressiontechniques to be used in order to meet the network bandwidthrequirements, or requires significant delays in the image display iflossless transmission is used. In addition, lossy compression on theorder required for real-time remote transmission, severely limitscomputer-based analysis, as well as human diagnosis, due to theartifacts associated with lossy compression techniques. Remote operationof a microscope also requires only a single user to use the instrumentat one time, requiring instrument scheduling and local maintenance ofthe instrument and the slides to be viewed.

The second option being investigated to overcome the limitations inheritin single frame imaging is a montage (or “virtual slide”) approach. Inthis method, a robotic microscope systematically scans the entire slide,taking an image at each “camera field” corresponding to the field ofview of the camera. Camera field and field of view shall hereinafter bereferred to as the “field.” The individual images are then “knitted”together in a software application to form a very large data set withvery appealing properties. The robotic microscope can span the entireslide area at a resolution limited only by the power of the opticalsystem and camera. Software exists to display this data set at anyresolution on a computer screen, allowing the user to zoom in, zoom out,and pan around the data set as if using a physical microscope. The dataset can be stored for documentation, shared over the Internet, oranalyzed by computer programs.

The “virtual slide” option has some limitations, however. One of thelimitations is file size. For an average tissue section, the datagenerated at 0.33 .mu.m/pixel can be between two and five gigabytesuncompressed. In an extreme case, the data generated from one slide canbe up to thirty-six gigabytes.

A much more difficult limitation with the prior systems is an imagecapture time problem. Given an optical primary magnification of twentyand a two-third inch coupled device or “CCD”, the system field of viewis approximately (8.8 mm times 6.6 mm)/20=0.44 times 0.33 mm. A standardtissue section of approximately 2.25 square centimeters, therefore,requires approximately fifteen hundred fields to capture an image of theentire tissue section.

Field rate, which is the amount of time it takes to capture an image ofa field and set-up the apparatus capturing the field for a followingimage capture, in montage systems is limited by three factors—cameraframe rate (the number of camera images acquired per second), imageprocessing speed (including any time required to read the camera data,perform any processing on the camera data prior to storage, and to storethe final data), and rate of slide motion, which is the time requiredfor the slide to be mechanically repositioned for the next imageacquisition. Given today's technology, the rate of slide motion is asignificant limiting factor largely because the existing imaging systemsrequire the slide to come to a stop at the center of each field tocapture a blur free image of the field.

For example, traditional bright field microscopic illumination systemswere designed to support direct visual examination of a specimen on thefield and therefore depend on a continuous light source forillumination. Continuous light, however, is a significant limitation fordigital imaging in that the slide, which must move to capture an entireimage, but must be stationary with respect to the camera during CCDintegration, thus moving the slide from the light. Moreover, slidemotion during integration results in a blurred image. Traditionalmontage systems, therefore, have had to move the slide (and stage) fromfield to field in a precise “move, stop, take image and move again”pattern. This pattern requires precise, expensive mechanics, and itsspeed is inherently limited by the inertia of the stage.

The three-dimensional characteristic of a typical tissue sample and theslide places additional limitations on the imaging system. Like alllenses, microscope optics have a finite depth of field—the distancewithin which objects will appear to be focused. A typical depth of fieldis about 8 microns for a 10.times. objective, and in general, as themagnification increases, the depth of field decreases. While microscopeslides are polished glass, the flatness of the slide can vary on theorder of 50 microns or more across the slide. The variations in thetissue sample thickness and any defects associated with placing thesample on the slide, such as folds in the tissue, also affect theoptimal position of the slide with respect to the imaging optics. Themagnitude of the optimal position and the limited depth of field of themicroscope optics require the focus to be adjusted as the system movesfrom field to field. The time to refocus the system at each field alsocontributes to the overall capture time of the montage image

Thus, a system is needed to reduce the image capture time. The systemmust also enable efficient and high quality imaging of a microscopeslide via a high-resolution slide scanning process.

FIELD OF THE INVENTION

The present invention relates to microscopic digital imaging of completetissue sections for medical and research use. In particular, itdescribes a method for high throughput montage imaging of microscopeslides using a standard microscope, digital video cameras, and anillumination system.

SUMMARY OF THE INVENTION

The present invention relates to a method and system for creating a highthroughput montage image of microscope slides. The system includes anoptical system, components that are used in a pre-scan processing, andcomponents for auto-focusing by enabling accurate focus control ofoptical elements without requiring the stage to be stopped and refocusedat each tile location. The optical system includes at least one camera,a motorized stage for moving a slide while an image of the slide iscaptured, a pulsed light illumination system that optically stops motionon the motorized stage while allowing continuous physical movement ofthe motorized stage, and a stage position detector that controls firingof the pulsed light illumination system at predetermined positions ofthe motorized stage. The components that are used in the pre-scanprocessing include an image-cropping component, a tissue findingcomponent and a scan control component. The image-cropping component andtissue finding component identify tissue regions on the slide in theoptical system and determine exact locations of tissue on the slide. Thescan control component uses information about the locations to generatecontrol parameters for the motorized stage and the camera. Thecomponents for auto-focusing include a focal point selection component,a focal surface determination component, and a scan component. The focalpoint selection component and the focal surface determination componentuse the control parameters to ensure that a high-quality montage imageis captured. The scan component is able to capture a high-qualitymontage image by maintaining motion of the motorized stage andsynchronization of the optical system. The scan component controls thestage position to maintain in-focus imaging during the scanning processwithout stopping the stage and refocusing at each location and fires apulsed-illumination source at the appropriate position to guaranteeimage alignment between sequential camera images.

Accordingly, it is a benefit of the invention that it provides amicroscopic imaging system for whole slide montage in which standardmicroscope optics, off the shelf cameras, a simple motorized stage, anda pulse light illumination system can be used to produce preciselyaligned image tiles, and acquire these image tiles at a speed limitedprimarily by the camera frame rate.

The present invention uses a strobe light, triggered by a direct Ronchiruler or other stage-positioning device, to produce precisely alignedimage tiles that can be made into a montage image of tissue sections ona microscope slide. Significantly, due to the short light pulse emittedby a strobe, clear images can be obtained without stopping themicroscope stage. This significantly increases the image throughputwhile decreasing the expense and precision required in the stagemechanics.

In one embodiment, a strobe arc is placed at the position of the lampbulb in a standard microscope system. The camera shutter is opened andthe strobe is fired in response to the position of the stage as reportedby a direct position sensor. If stray light is minimized, the cameraexposure can be much longer than the strobe flash, allowing low costcameras to be utilized.

It is another benefit of the invention to significantly increase theimage throughput of a tiling image system by allowing, through the useof the strobe light, continuous motion of the slide under themicroscope. The inventive system thus eliminates the need to stop themicroscope stage to capture an image

It is another benefit of the invention to reduce the demands of camera,stage, and strobe synchronization by controlling the firing of thestrobe light based on direct stage position feedback, therebysubstantially reducing the mechanical precision required of the stageand camera components.

It is another benefit of the invention to use a pre-scan process appliedto a macroscopic image of the entire slide, to guide a high-resolutionslide scanning process and ensure high-quality images of the entirespecimen are acquired. The pre-scan process includes an image croppingcomponent, a tissue finding component, a scan control component, a focuspoint selection component, a focal surface determination component, anda scan component. The image cropping and tissue finding componentsidentify interesting regions on the slide to be scanned. The focus pointselection and focal surface determination components ensure that a highquality image is captured by the scanning process, by enabling accuratefocus control to be maintained

It is another benefit of the invention to use a high-resolution slidescanning process to control the operation of the motorized stage, cameraand strobe illumination. This process utilizes information gathered bythe pre-scan process, namely the imaging regions and focus parameters,to control the positioning of the stage to image only the regions ofinterest and to ensure the individual images are well aligned and infocus.

Additional features and advantages of the invention will be apparentfrom the description that follows, or may be learned by practice of theinvention. The objectives and advantages of the invention to be realizedand attained by the microscopic image capture system will be pointed outin the written description and claims hereof as well as the appendeddrawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are included to provide a furtherunderstanding of the invention and are incorporated in and constitute apart of this specification, illustrate embodiments of the invention thattogether with the description serve to explain the principles of theinvention

FIG. 1 is a partial isometric view of an embodiment of an imagingapparatus of the present invention;

FIG. 1 a is a front view of the apparatus of FIG. 1;

FIG. 1 b is a side view of the apparatus of FIG. 1;

FIG. 1 c is a top view of the apparatus of FIG. 1;

FIG. 2 is a macroscopic image resulting from operation of an embodimentof the cropping component;

FIG. 3 illustrates a result of the tissue finding component;

FIG. 4 is an overlay of FIGS. 2 and 3 illustrating the regions of theslide to be imaged;

FIG. 5 illustrates a result of the focus point selection component on asample image;

FIG. 6 is a generated three-dimensional data set for the image of FIG.5; and

FIG. 7 is a set of charts graphically depicting steps implemented in anembodiment of the inventive system.

DETAILED DESCRIPTION OF THE DRAWINGS

Reference will now be made in detail to the preferred embodiments of thepresent invention, examples of which are illustrated in the accompanyingdrawings. The following paragraphs describe the functionality of theinventive system and method for focus controlled, high throughputmontage imaging of microscope slides using a standard microscope,camera, and a motorized mechanical stage.

FIG. 1 illustrates a first embodiment of the imaging apparatus of thepresent invention. FIGS. 1 a-1 c illustrate front, side, and top views,respectively, of the imaging apparatus illustrated in FIG. 1. It shouldbe noted that not all components that may be included in an imagingapparatus are illustrated in FIGS. 1-1 c. For example, a stage supportthat attaches the stage to the imaging apparatus is not illustrated soas not to obstruct the view of other components. In this embodiment, aslide 112 to be imaged is placed on a thumbnail imaging position in aslide holder on a motorized stage 102. A single frame image containingthe entire slide may be taken with a macro camera 106. One skilled inthe art will recognize, however, that a suitable image may alternatelybe captured by combining multiple images taken by a microscopic camerasuch as microscopic camera 104. Microscopic and macroscopic images mayfurthermore be captured by use of two separate cameras or through theuse of a single camera. The low-resolution image may be analyzed bysoftware components, described below, to determine the locations oftissue on slide 112. This information can then be used to generatecontrol parameters for stage 102 and microscopic camera 104 to ensurethat the scanning process captures high quality images of only thetissue regions, substantially reducing the time to scan an averageslide.

One skilled in the art will recognize that, although capturing thesingle macroscopic image saves time, it is not necessary for theoperation of the invention. However, where a macroscopic image is to beused, multiple macroscopic images may be required to generate controlparameters to the accuracy required based on the ratio of themacroscopic to microscopic magnifications and the camera specificationsof each camera, if separate cameras are utilized.

Once the regions to be microscopically imaged are identified, the slideis scanned under the microscope optics. To facilitate rapid imaging ofthe slide and to avoid the stop image reposition delays associated withtraditional imaging systems, a high-speed strobe light is used tooptically stop the motion of the stage, and thus the slide specimen,while allowing continuous stage motion. It should be apparent to oneskilled in the art, that any pulsed illumination system may be used inplace of the high-speed strobe light. To eliminate overlap or missedtissue between the microscope images, precise alignment of the stage andcamera, along with accurate stage positioning, and camera and strobesynchronization, are required. To reduce the camera precisionrequirements, a direct stage position sensor is used to control thefiring of the strobe, and thus the camera exposure.

In this fashion, the camera can be operated with a long exposure windowin comparison to the very short strobe flash, allowing lower costcomponents, specifically the stage and camera, to be utilized.

Specifically in an embodiment illustrated in FIGS. 1-1 c, a pre-scanprocessing of the low-resolution or thumbnail image includes an imagecropping component, a tissue finding component, a scan controlcomponent, a focus point selection component, and a focal surfacedetermination component. Those pre-scanning components may then befollowed by a scan component. The image cropping component and tissuefinding component identify tissue regions on the slide to be scanned.The scan control component generates the necessary control parameters toscan only the regions of interest under the microscope optics. The focuspoint selection component and a focal surface determination componentensure that the scan component captures a high-quality montage image, byenabling accurate focus control of the optical elements withoutrequiring the stage to be stopped and refocused at each tile location,substantially reducing the acquisition time.

One step in processing the thumbnail image consists of flat-fieldcorrecting the macroscopic thumbnail image using a similar image thatmay have been obtained from the same camera and a blank slide. Thisremoves any spatial light anomalies from the thumbnail image, which mayreduce the efficiency of the tissue finding component. Depending uponthe format, or size, of the camera and the aspect ratio of the slide, aportion of the image may contain non-slide objects such as a slidecarrier (not shown in the figures). To remove these features, thethumbnail image may be cropped to remove non-slide objects, thusretaining only the slide information.

The image cropping may be accomplished via a two-pass process. The firstpass in such a process determines an approximate location of the slideboundary, and the second pass fine-tunes this estimate. The search forthe boundary is conducted over upper and lower intervals correspondingto the regions expected to contain the upper and lower slide edges,respectively. The slide or region of interest may be assumed to bepositioned near the center, vertically, in the thumbnail image. Theportion of the image falling outside of the identified slide boundary isremoved. It should be noted that the cropping component and each of theother components described herein may operate on either a grayscale orcolor image. The image may also be cropped at the estimated edgelocations, and then uniformly reduced in size to produce a smallthumbnail image of the slide for rapid, visual slide identification.

Because the slide may not be oriented perfectly horizontally in theoriginal thumbnail image, the identified slide edges are likely to lieat an angle. Thus, even after cropping, there may be remnants of theslide edges or cover slip in the cropped image. Therefore, the imagecropping component attempts to identify pixel blocks that likely containthese remaining edges and flags these blocks as edges that will not beconsidered for high resolution imaging by the tissue finding component.

The resulting cropped image generated by the image cropping componentmay serve as an input to the tissue finding component. This componentlocates regions in the thumbnail image that contain tissue of interestto a specialist. In order to minimize the time and storage spacerequired to accomplish high-resolution slide imaging, the inventivesystem may capture only those regions of the slide that contain tissue.This approach may be facilitated by identifying regions containingtissue in the thumbnail image

The tissue finding component identifies tissue regions via a sequence offilters that incorporate knowledge of the typical appearance andlocation of tissue and non-tissue slide regions. Initial filteringsteps, in one embodiment, convert the image to a grayscale image andanalyze the mean and standard deviation of the local pixel intensities.Pixel mean intensities may be used to differentiate tissue-containingregions from blank and other non-tissue regions, such as thosecontaining the slide label or other markings. The standard deviationdata may represent the amount of variation in pixel intensity values andthus is a good indicator of the border between tissue and the blankslide. The mean and standard deviation data is combined to generate athreshold value that is used to make an initial classification of tissueversus non-tissue. Subsequently, morphological filters may be applied torefine the classification based on the size and position of neighboringgroups of potential tissue pixels.

The embodiment described above uses the mean and standard deviation ofthe local pixels as the basis for detecting regions of interest. Oneskilled in the art will recognize, however, that other imagecharacteristics can also be used to identify the specimen from non-itemsof interest such as dust and scratches. That embodiment may also processa grayscale macroscopic image. It should be noted, however, that thepixel intensity differentiation tools described herein can be applied toeach of the color components (traditionally, red, green and blue) of acolor image in addition to being applied to a grayscale image.Additional processing tools can also be applied between color componentsto refine the tissue finding accuracy and to remove features such aslabels and writing that are not critical to the application, or toselect user defined areas of interest to be scanned, such as regionscircled by a purple marker.

The filters, which comprise the tissue finding component, process thepixels of the cropped grayscale thumbnail image in groups thatcorrespond to slide regions, or tiles, that can be imaged individuallyduring the high-resolution scanning process. These filters ensure thattiles only partially filled with tissue are classified astissue-containing tiles. The final output of the filter sequence is atiling matrix, the value of which indicates which tiles should beimaged. The tiling matrix subsequently guides the high-resolutionscanning process.

An example of the image cropping and tissue finding processes are shownin FIGS. 2, 3 and 4. FIG. 2 illustrates the macroscopic image afterflat-field correction and image cropping. FIG. 3 illustrates the resultsof the tissue finding component. The resulting tile matrix shown in FIG.3 has a one-to-one correspondence to the field of view of themicroscopic camera. White pixels (binary 1) signify fields to becaptured and imaged, and black pixels represent regions not to image.FIG. 4 illustrates an overlay of FIGS. 2 and 3 representing the sectionsof the slide to be imaged. For the application depicted in FIGS. 2-4(anatomical pathology), it may be important to image all suspect regionsthat may contain tissue so conservative criteria are used in the tissuefinding component, resulting in cover slip edges and writing etched intothe slide to be identified as to be imaged. The savings in theacquisition time is represented by the ratio of the white to black areasof FIG. 3. For this image, only 53% of the slide region is to be imaged,including the label and cover slip edges, and etched writing on theslide.

At the completion of the tissue finding component, the scan controlcomponent interprets the tissue finding tile matrix (FIG. 3) andtransposes the positions into actual stage coordinates for themicroscopic imaging. A program running on a host computer controls theoperation by communicating with a stage controller and microscopiccamera 104. Actual scanning can occur in any fashion such as by rows orcolumns, or in a step fashion to image neighboring areas.

To achieve good focus for an entire slide, the surface that bestrepresents the focal position of the sample with respect to the opticalelements may be determined and used to automatically control the focusas the tissue is scanned under the microscope optics. These steps mayfurthermore be completed under the focus point selection component andthe focus surface determination component.

In one embodiment, the focus point selection component evaluates thetissue regions of the thumbnail image and selects several points onwhich to initially focus the microscope optics. In this embodiment, thatfocus point selection is based on pixel intensity and distribution.Initially, tissue containing pixels or groups of tissue containingpixels are considered. Those pixels or pixel groups will be referred tohereinafter as regions. The regions that will be focused upon, or focusregions, are then selected based on the relative contrast between theintensity of those regions within the thumbnail image and thedistribution of those regions within the tissue containing portions ofthe image. It will be recognized that other intensity and/ordistribution selection criterion may be utilized to select focusregions. After the focus regions are identified, a normalized focussurface such as a plane or curve may be drawn through the focus pointsand an equation, model, curve, or other method for describingappropriate focal lengths along that surface can be obtained. Properfocus for each region to be captured by the high-resolution scan maythen be calculated by focusing on, for example, the plane or curvedsurface.

Selecting the appropriate focus region based on relative contrastimproves the likelihood that regions that contain tissue will beselected over a method of picking focus regions based on a pre-selectedgrid-pattern. Selecting regions based on their overall distribution withrespect to the tissue coverage area provides assurance that theresulting plane or surface will be representative of the entire slideand not limited to a small portion of the tissue, as could occur ifselection were based solely on the relative contrast of pixels.

To select focus regions based on contrast, the present invention mayselect a number of focus regions having a desired contrast quality. Forexample, six to ten of the darkest regions may be selected from aback-lit image under the assumption that dark regions contain a largeamount of tissue.

Distribution refers to the overall distribution of regions with respectto the tissue coverage area. Because a surface defined by the focusregions is the basis for maintaining the focus across the entirespecimen during scanning, it is beneficial to have focus regionsdispersed across the specimen rather than being grouped in closeproximity to one another in one or more areas of the specimen. In thatway, every point to be scanned will be close to a focus point. Thus, thescanned optical position at each point, as defined by the surface,should be very nearly the optimum in-focus position for that point.

The focus point selection component may, for example, assure that atleast one focus point is located within each of a number of pre-selectedareas of the tissue. Those areas may, for example, be separate pieces oftissue on a slide. If a focus point does not exist on each pre-selectedarea, the focus point selection component may select additional regions,for example, having the desired contrast quality, until at least onefocus point is identified on each pre-selected area. The number of datapoints required will depend on the actual three-dimensional structuredefined by the specimen and slide, and the geometrical dimension of thesurface to be fit. Once the surface has been determined, an errorfunction can be calculated to determine the fit accuracy. For example,the mean square error of each selected focal region may be calculated todetermine how much error exists between the surface fit and each focusregion. If that error is greater than a predetermined acceptable errorlevel, at one or more points, additional data points may be added to thecalculation and/or points that have large errors may be eliminated andthe surface may be recalculated under the assumption that the pointshaving excessive errors were anomalies. A surface may then be fitted tothose points to define a focal surface to be utilized when scanning thetissue.

In alternative embodiments, the focus points are either user definablethrough an input file or through a suitable user interface. In addition,for cases where the specimen locations are reproducible on the slides,the focus points can be predefined and repeated for each slide withoutthe use of a macroscopic image or any preprocessing to find the tissueregions.

Once selected, each focus position is placed under the microscope opticsin the focal surface determination component, and an auto-focus programdetermines the best-fit focus at each position. This generates athree-dimensional data set corresponding to the optimal specimendistance at each stage location. These data points are used as input toa surface fitting routine that generates the necessary controlparameters for the slide scanning process.

At the completion of the focus point selection, the tissue informationand the surface parameters are passed to the scan control component.This component is responsible for the motion of the stage andsynchronization of the microscopic imaging system during montage imageacquisition. To achieve accurate, well-aligned tiled images, thespecimen may be positioned such that each camera image is aligned withinthe equivalent single pixel spacing in real or stage space (camera pixelsize divided by the overall optical magnification). This usually entailsa stage step of ox and by where each step is directly related to theeffective pixel size, which may be expressed in terms of camera pixelsize/optical magnification, and the number of image pixels in the x andy directions respectively. For example, a 1300.times.1030 pixel, 10.mu.m square pixel camera operated at 20.times. magnification results in.delta.x=10 .mu.m*1300/20=650 .mu.m and .delta.y=10 .mu.m*1030/20=515.mu.m. To maintain focus during the scanning process, the stage may bepositioned at the proper focal position as determined by the focussurface parameters: z.sub.ij=f(x.sub.i, y.sub.j), where z.sub.ij is thevertical position of the slide with respect to the optical components,f(x.sub.i, y.sub.j) is the function used to represent the best focussurface, and x.sub.i and y.sub.j are the positions of each camera imagein the x and y axes, respectively. The camera image positions can beexpressed as linear relations between the starting position of the stage(x.sub.0 and y.sub.o) and the step size associated with each imagedimension: x.sub.i=x.sub.0+i*.delta.x and y.sub.i=y.sub.0+i*.delta.y.

Image montage scanning is traditionally accomplished by either scanningby rows or columns. Assuming that tiling is completed by scanning acrossa row in the x-direction and stepping vertically in the y-directionafter each row has been scanned, the stage is simply positioned at theappropriate position given by z.sub.i,j=f(x.sub.i, y.sub.j). Thereafter,the stage is stopped and an image is acquired. If imaging isaccomplished during continuous motion of the stage in the x-directionvia a line scan camera or alternative imaging arrangement, the verticalvelocity as a function of focal position x.sub.i and time, can becomputed from the partial derivative of the focal surface:V.sub.z(v.sub.x, y.sub.j)=..delta.f(x.sub.i, y.sub.j)/.delta.x*v.sub.x,where V.sub.z is the velocity of the vertical position of the stage andv.sub.x is the velocity in the x-direction. The velocity of the verticalposition of the stage can be used to control the optical position andmaintain focus as images are acquired continuously across the row.

FIG. 5 represents the results of the focus point selection component.This figure shows the thumbnail or macroscopic image of the region to bescanned. The light spots 504 overlaid on the specimen 506 represent thepositions selected by the focus point selection component. Thesepositions are placed under the microscope and auto-focused on eachlocation. FIG. 6 illustrates the three-dimensional data set generated byfocusing on each of the focus points of the specimen depicted in FIG. 5.For this slide, the best fit was planar, z(x,y)=dz/dx x+dz/dy y+z0,where dz/dx (dz/dy) is the slope or pitch of the plane with respect tothe x-axis (y-axis) and z0 is the vertical offset of the plane withrespect to the z-axis. The best fit parameters for the specimen of FIG.5 are also shown in FIG. 6.

At the completion of the pre-scan processing, the tile matrix and thestage control parameters are passed to a scanning-process controlprogram. The scanning process control program controls the operation bycommunicating with a stage controller, a stage position sensor, a cameraand the strobe firing circuitry. In the invention, the computer programcontrols the operation of stage 102, camera 104 and strobe 108illumination. The actual slide scanning can be automated to image entireslides, image only a portion of the slide or use a user-interface toallow the user to select the regions to be imaged. Once a region hasbeen selected for imaging, the program then controls the operation bycommunicating with a stage controller, a stage position sensor, camera104 and strobe firing circuitry 108. Preferably, tiling is performed bymoving stepwise along the short axis and with continuous motion alongthe long axis. In other words, tiling is done one row at a time. Forthis reason, a stage position is monitored and controlled differentlyalong each stage axis. Along the short axis of the slide, the stageposition is monitored and controlled, by the program, directly throughthe stage controller. Along the long axis, however, the stage positionis monitored by a direct stage position sensor, which can be separate orpart of the overall stage control circuitry.

In another embodiment, a Ronchi ruler attached to stage 102 is used forthe stage position sensor. Those skilled in the art will recognize thatany position sensor may be used in the invention. This sensor can beexternal to the stage controller or the positional information can beacquired directly from the stage controller with or without feedback.

For reference, a Ronchi ruler is a pattern of alternating light and darkbands, equally spaced along a substrate, typically either glass orplastic. A position sensor based on the Ronchi ruler utilizes a lightsensor that is mechanically isolated from the ruler. As the ruler passesunder the sensor, a series of electronic pulses is generatedcorresponding to the alternating light and dark bands of the ruler.These pulses can be used to monitor the position and direction of stage102.

Based on the magnification of the optics and the camera utilized, strobe108 is fired whenever the position sensor determines stage 102 has movedinto the field of view of the next tile to be captured by the camera104. The system continues to capture image tiles with precise alignment,until the images of all desired files have been captured or thecontrolling program tells the system to stop. At the end of the captureprocess, the slide is removed and another slide can be inserted. Withcurrent technology, the rate-limiting step for image capture utilizingthe present invention is the data transfer period in the camera.

FIG. 7 illustrates the signals of camera 104, stage 102, opticalposition detector, and strobe 108. Note that in FIG. 7, graphs 202 and204 the signals from the optical position detector represent motion ofstage 102, so their timing will vary depending on the speed of the stagemovement. Where the system is triggered by the location of stage 102 asreported by the optical position sensor, precise movement of the stagemovement is not necessary, allowing for the use of low cost stages 102.

The system can be run in a variety of modes, depending on how the camerais controlled. In one embodiment, the stage location, as sensed by aposition sensor, fires both the camera 104 and the strobe 108 asindicated by the two traces at 204. In an alternate embodiment, camera104 is free running and only strobe 108 is fired by stage position asindicated by a single trace at 204. This mode does not depend on uniformmotion of stage 102 over the area imaged, because the strobe pulse ismuch shorter than the integration time of the camera 104, wherein theintegration time is a time during which the camera is receiving animage. As long as the correct stage position is reached anytime withinthe integration time of camera 104, an excellent, well aligned imageresults.

Firing strobe 108 based on direct position information differs from themore traditional application of strobe photography. In traditionalstrobe photography, a strobe and camera are synchronized in time, andpositional information regarding the objects being imaged can beinferred from the relative position within the image. When the presentinvention is operated in a mode wherein the position feedback controlsboth camera 104 and strobe 108, and camera 104 is not free running, eachcamera frame corresponds to an equally spaced positional change,independent of the stage velocity (speed and time variations in thespeed). In the case that camera 104 is free running, the stage speed hasto be matched to the camera frame rate only to the accuracy such thatthe strobe pulse does not fall outside the exposure window. The relativetime within the exposure window is irrelevant.

As will be understood by one skilled in the art, while the presentinvention describes a microscopic optical arrangement, the invention canalso be applied to other optical imaging, inspection and illuminationsystems that are used for building up an image by matching the stagespeed with the camera speed.

The foregoing description has been directed to specific embodiments ofthis invention. It will be apparent, however, that other variations andmodifications may be made to the described embodiments, with theattainment of some or all of their advantages. Therefore, it is theobject of the appended claims to cover all such variations andmodifications as come within the true spirit and scope of the invention.

1. An imaging apparatus, comprising: a motorized stage; a camera focusedrelative to the motorized stage; and a processor coupled to the camera,wherein the processor contains instructions which, when executed by theprocessor, cause the processor to: capture a low resolution image thatis incident on the camera, wherein the low resolution image includes aplurality of pixels, the pixels having a characteristic; establish thecharacteristic for each pixel; determine which of the pixels contain atarget image based on the characteristic of the pixels and establish atarget area that includes those pixels; transpose the position of thetarget area into a plurality of stage coordinates; capture a highresolution image that is incident on the camera at each of the stagecoordinates, wherein the processor determines which pixels contain thetarget image based on a relative intensity of the pixels and further:determines a mean intensity of the pixels; compares the intensity ofeach of the pixels to the mean intensity; divides the pixels into agroup of non-target image pixels having high intensities and a group oftarget image pixels having intensities lower than the high intensities;determines, for the pixels, an intensity standard deviation thatprovides an amount of variation in pixel intensity; compares theintensity of each pixel to the intensity standard deviation; and dividesthe pixels into a group of non-target image pixels having low standarddeviations and a group of target image border pixels having standarddeviations that are greater than any of the low standard deviations. 2.The imaging apparatus of claim 1, wherein the characteristic includespixel intensity.
 3. The imaging apparatus of claim 1, wherein thecharacteristic includes pixel color.
 4. The imaging apparatus of claim1, further comprising a pulsed light directed toward the motorizedstage.
 5. The imaging apparatus of claim 1, further comprising a stageposition sensor adjacent the motorized stage.
 6. An imaging apparatus,comprising: a motorized stage; a camera having a lens directed towardthe motorized stage; and a processor coupled to the camera, wherein theprocessor contains instructions which, when executed by the processor,cause the processor to: select at least three points of a sampleadjacent the motorized stage depending on a characteristic of an imageof the sample at those points, wherein the selecting at least threepoints dependent on a characteristic of the image of the sample at thosepoints includes selecting the darkest regions; determine a stageposition for each of the selected points; focus the camera on each ofthe selected points; determine an object distance from the camera lensto the sample at each of the selected points; develop a normalized focussurface based on the stage position and the object distance for theselected points, wherein when the processor selects at least threepoints of a sample adjacent the motorized stage, the processor further:determines a distribution of the at least three selected points withinthe sample; determines whether at least one of the selected points lieswithin each of at least two predetermined areas; and selects additionalpoints until at least one of the additional points lies within eachpredetermined area.
 7. An imaging apparatus, comprising: a motorizedstage; a camera having a lens directed toward the motorized stage; and aprocessor coupled to the camera, wherein the processor containsinstructions which, when executed by the processor, cause the processorto: select at least three points of a sample adjacent the motorizedstage depending on a characteristic of an image of the sample at thosepoints, wherein the selecting at least three points dependent on acharacteristic of the image of the sample at those points includesselecting the lightest regions; determine a stage position for each ofthe selected points; focus the camera on each of the selected points;determine an object distance from the camera lens to the sample at eachof the selected points; develop a normalized focus surface based on thestage position and the object distance for the selected points, whereinwhen the processor selects at least three points of a sample adjacentthe motorized stage, the processor further: determines a distribution ofthe at least three selected points within the sample; determines whetherat least one of the selected points lies within each of at least twopredetermined areas; and selects additional points until at least one ofthe additional points lies within each predetermined area.
 8. Theimaging apparatus of claim 7, wherein the selecting points dependent ona characteristic of the image of those regions includes selecting pointshaving a high contrast relative to the regions.